1. Field of the Invention
The present invention relates to an apparatus for processing an image of a face and more particularly to an apparatus of processing an image of a face for processing the image of a driver's face taken with a camera in such a manner as to extract a characteristic feature area of the face and then detect a driving condition of the driver on the basis of the condition of the extracted characteristic feature area.
2. Description of the Related Art
An apparatus of processing an image of a face is known in the art, that extracts an eye, which is one of characteristic features of a face, by processing the image of a driver's face taken with a camera disposed in a car, thereby detecting a driving condition such as looking off or dozing. In one known apparatus, an eye is extracted, by means of template matching, directly from a face image in the form of a halftone image without being converted into another form (Japanese Patent Laid-Open No. 6-255388 (1984)). In another known technique, an eye is extracted by detecting the darkest point of a face image (Japanese Patent Laid-Open No. 6-266981 (1984)). In still another known technique, a halftone image is converted into a binary (two-level) image, and then an eye is extracted by detecting a black area inside the face contour of the binary image (Japanese Patent Laid-Open No. 6-32154 (1994)). Of these conventional techniques, the technique disposed in Japanese Patent Laid-Open No. 6-32154 (1994) will be described in greater detail below.
FIG. 42 is a flow chart of the process of detecting the condition of a driver according to the technique disclosed in Japanese Patent Laid-Open No. 6-32154 (1994). FIG. 43 is a schematic representation of the process of converting a face image into a binary form according to this technique.
In the first step S88 shown in FIG. 42, an image of a driver's face is taken with a camera and the resultant halftone image signal is applied to image input means (not shown). The image signal in analog form is then converted into digital form, and the resultant digital halftone image is stored in a frame memory (not shown) in step S89.
In step S90, the image data is read from the frame memory, and converted by binarization means (not shown) into a binary image with respect to a proper threshold. In step S91, a starting line of scanning the face image in a horizontal direction (also referred to as Y-direction) is defined, and white-level picture elements are searched for by scanning the face image in the horizontal direction starting from the starting line. In step S92, the number of successive white-level picture elements is counted. In step S93, the vertical boundary of the face image is recognized by detecting the ends of an area including the greatest number of successive white-level picture elements.
In step S94 the Y-coordinates of an eye searching area are defined on the basis of the vertical boundary of the face image determined in step S93. In step S95, a starting line of scanning the face image in a vertical direction (also referred to as X-direction) is defined. In step S96, black areas containing successive black picture elements in the eye searching area are detected by scanning the face image starting from the starting line defined above. In step S97, an eye area is identified judging from the positional relationship among the detected black areas and also from the number of black picture elements counted in the vertical direction.
Finally in step S98, instantaneous eye detection means (not shown) detects open-and-close motion of an eye by detecting the number of black picture elements counted in the vertical direction within the eye area identified in the previous step. In step S99, doze judgement means (not shown) determines whether the driver is dozing or not on the basis of the information of the open-and-close motion detected in the previous step.
In the above-described conventional technique, however, it is required to always illuminate the face by a near-infrared light ray with a high intensity to obtain a stable binary image. In particular, in daytime operation, a very high intensity of illumination is required to reduce the disturbance due to the near-infrared components of sunlight.
In some cases, the face image is taken under illumination of sunlight without using artificial illumination which needs high electric power. However, in this technique, the illumination of sunlight is disturbed, during running of a car, by the direction or altitude of the sun and other circumstances such as sunlight filtering down through trees. As a result, a face can be shaded and thus the face image cannot be converted into a correct binary image, which brings about a problem in extracting an eye. In the example shown in FIG. 43, the face of a driver is illuminated by sunlight coming obliquely from the front of the driver wherein the upper half of the face is shaded by the frame or sun visor of a car. If the face image is converted into a binary form according to the conventional technique, a great difference occurs in picture element signal level between the upper half and lower half of the face, and thus the threshold value in the binary conversion process becomes high due to the bright portion of the face image. Thus, in the example shown in FIG. 43, although nares and a line between lips can be detected, the upper half part including eyes, hair, and eyebrows is recognized as a single black area and thus it is impossible to extract an eye area.
On the other hand, in a technique of extracting an eye from a halftone image without using a binary image, a high-capacity frame memory is required for storing a halftone image. However, this results in high cost. Furthermore, the dealing with the halftone image requires a rather long time, and it is difficult to achieve a high enough speed in real time operation.
Furthermore, in the technique of detecting an eye using the face contour, if the background is bright, the face contour cannot be extracted correctly. In the technique based on the pattern matching, variations in the shape or positions of eyes or glasses lead to a difficulty in the pattern matching process. On the other hand, in the technique based on the detection of the darkest point, it is difficult to distinguish a pupil of an eye from other black areas such as a mole.